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Abstract 


In many technology related courses in addition to the new content, previously learned knowledge 
about science and science related supportive tools are simultaneously studied. In the field of 
applied sciences, the aim is to apply technology applications derived from these scientific methods 
and models to the operating environment. From the point of view technical disciplines in cadets’ 
education are important but narrow in scope, so there is a need to look for solutions that meet 
the educational goals of the local community. In this research, it is considered to build a broader 
assessment, to support learning to be motivating, and also to guide teachers at their work. This 

empirical research consists of findings from three successive courses in the sensor technology 
education for cadets. Examination structure is analyzed according to the test scoring and student 
success. As a conclusion, a course of specific mix of quiz can be recommended. Moreover, the 
principles of the presented form-based test are transferable to electronic assessment environments. 

Keywords: structure of assessment, summative test, technology education. 


Introduction 


The National Defense University (NDU) is a training institution responsible 
for educating the future leaders of Finland’s armed forces and Border Guard. Due to a 
clear link to work life, training can emphasize professional aspects and, in particular, 
the practical skills required in the first job. Nevertheless, the education should take into 
account academic research and general capacity building for lifelong skill development. 
Also, efforts have been made to reform the structure of the training for organizational 
needs or for the external reasons. The latter include the Bologna process or the Karvi 
audit (Karvi, 2017). This study discusses the teaching and development of specific 
surveillance and target accusation course. Findings from previous implementations of 
the course have been discussed earlier (Rissanen, 2010, 2014). This research focuses 
on students’ assessment in three successive courses in years 2014-2016. As a formative 
assessment, cadets’ learning was monitored through colloquial discussions and short 
exercise assignments related to lectures. In the end of the course a summative test was 
held for grading. 


Evaluation as Part of Teaching 
University learning is aimed to a permanent change that can be derived from the 


student’s personal work during the course. In general, the first step is to use assessment 
tools to collect information on student learning (measurement) and then to evaluate each 
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student’s learning level (assessment) (Snowman, McCown, & Biehler, 2008). In the 
military context, most of the courses include a number of learning objectives, some of 
which can be considered as tacit knowledge, which is difficult to measure without real 
[military] activity and the real threats associated with it. Anderson’s and Krathwohl’s 
(2001) taxonomy classifies learning types with verbs, separating lower-level skills 
(remember, understand) and higher levels (applying, analyzing, evaluating, creating) 
from each other. The basic level of military technology knowledge is based on the 
transformation of remembrance into understanding and later on to the ability to apply 
key information. Technological education defines the process of technology and leads to 
identify relevant problems for solving. 

Over its basic function an exam could also support other educational aspects such 
as long-term learning, creative thinking, and motivation. In that task it would be fine to 
provide proof of competence and assessing its level using a real operating environment. 
However, larger groups of students for whom applied exercises are limited due to 
resource allocation must be satisfied with traditional tests. A versatile exam structure can 
motivate and inspire students. In practice, several types of exams are required. Often, 
the course content controls which types of tests produce the best overall outcome. In 
practice, the aim is to find a solution that reliably measures learning but is equally easy 
for an inspector to write reviews and grades for students. The development of evaluation 
must be seen as part of better education. 

Research question I. What new information did the analysis of the case course provide 
for the development of the assessment? 

Research question 2. How can the results be taken into account in the development of 
the evaluation? 


Research Methodology 


The focus of action research is on the influence of action and on the participation 
and involvement of researchers in the organization’s everyday life. Attendance is 
combined with subject analysis and influencing into practice. Kemmis and McTaggart 
(2000) emphasized that in reality the process may not be as straightforward as sequential 
parts of independent design, operation, observation and reflection. The described modules 
may overlap, and the original plans may become obsolete based on experience and new 
information. In O’Leary’s (2004) model, action research was seen as an experimental 
learning approach, with goals including refining the needs of the methods, knowledge, 
and interpretations based on the understanding of previous cycles. In this research three 
successive cycles were given. Interventions were made to proceed towards a better 
assessment of learning outcomes. 


Research Setting 


The exams should measure the student’s skills and also support students’ learning 
in the long term, provide the teacher with information on how to continue the course, but 
also lead to the plans that balance the workload of the teacher. Occasionally, students’ 
and the teacher’s perception of the learning goal may differ. For this reason, in the case 
of large entities, the teacher strives to focus on key areas. Guskey, (2000) has pointed out 
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that lazy students may complain that the tips of the offered reading list did not match so 
that it would be easy to complete the whole course, and even with great honors. For this 
reason, giving straight tips to the test is not entirely meaningful. 

A well-prepared assessment reflects the concepts, knowledge and skills that the 
teacher has emphasized in the classroom, in the textbooks, and in the learning material 
portal. The selected criteria for assessing student performance must be adapted to reflect 
the chosen approach. Under such conditions, an exam based on learning would not 
surprise students. Desired learning goals must be achievable for students and the course 
evaluation is one of the elements in the process. Instead of “teaching to the test’, one 
should evaluate what is taught and by what means. A good overall assessment of the 
situation serves as a meaningful source of information for teachers, helping them identify 
what they taught well and what they need to do (better) next time. (Guskey, 2003) 

Occasionally, the estimate of the development may be a simple summary of how 
many students did not complete the performance at each assessment point or did not 
meet the pre-set criteria. When looking at these results, it is necessary to look at the 
quality of the assessment elements or criterion. Is the question ambiguous or unclear? 
Did the students misinterpret the question or what was meant to be measured? 


Structure of the Final Exam 


During the study the summative test consisted of 5 elements with light variation 
in amount and theme. The elements were: 

e Explain concept, meaning and application which can be expressed in no more 

than five sentences, picked up from the content and lessons. 

e Short answer. (This was wider than just explain concept, e.g. compare two 

items). 

e Claim or statement (True / false). 

e Task partition: Explain on the basis of the image. 

e Task part: -essay or calculation task. 

Yearly the exam was repeated at least twice based on each year’s lecturing 
material. For clarity reasons major part of the tasks were repeated year after year. Even 
though theme was not changed, light modifications were done e.g. for clarity reasons. 
As an example, two clear images types were utilized in the explain image section. The 
first one was the diagram of the Planck model of black body radiant heat as a function 
of source temperature. Usually the image contained four curves in the logarithmic scale. 
Identifying the image and naming the items using the Wien’s displacement law was 
enough for an accepted solution. The model answer included extra points for additional 
information. The other commonly asked image type showed atmospheric attenuation 
from UV radiation up to 15 um. The aim here was to remember this very often repeated 
concept and to verify scale for visible light in the spectrum and also the so-called 
atmospheric windows. Both tasks required some insight, but the tips students could 
utilize were provided as correct information within the exam leaflet’s other parts. On 
the other hand, a broader essay task without quantitative delineation produced a lot of 
text and thus it was a burden to the teacher. Therefore, lot of preset structural order 
requirements and guidance were set to normalize answers. The alternative calculation 
was simple to grade but turned out less popular among students. 
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Research Results 


The course and the examination were considered difficult. According to some 
students, it did not test every major theme from the course description. To analyze 
interventions, the student success in each section of the exam was listed and there 
after compared in three successive lecturing years. The listed details needed additional 
analysis from the instructors’ observations and students’ evaluation of teaching (SET). 

Table | shows the six sections of the final exam in the technology course. Question 
types were listed as the guiding column. Each row showed: student group’s average 
record in the specific question section, next the maximum score, and finally relative 
success (%). Because there were variations in the number of subtasks and the detailed 
structure of tasks in each year, only the relative values were compatible. The percentage 
value was determined by dividing the average score by the maximum reachable value 
given by the assessment model’s data. 


Table 1. Test scoring and student success in three different years. 


2014 2015 2016 
Question types Average | max | (%) | average | max | (%) | average | Max (%) 
Explain concepts 49 6 82 4 7 53 45 7 64 
Short answers 2.2 4 54 3.2 4 81 5.7 6 96 
Claims(true/false) 12.5 16 78 15 20 76 18.2 22 83 
Picture 1 1.3 4 31 2.3 6 37 34 7 48 
Picture 2 2.1 4 53 4 6 62 3.7 8 46 
Essee or calculus 74 12 59 5.7 10 57 6.1 12 56 


Major finding was that a broader, detail full, and scaled-up test seemed to collect 
better results of knowledge even when the student’s knowledge includes a non-organized 
set of individual knowledge and skills. Set of individual students’ previous success was 
compared to the answers in this exam. In this case, a student who was uncertain about 
his / her knowledge, as the test progressed, worked on his knowledge. Therefore, he was 
able to exceed the critical limit to obtain such a score that he has passed the exam or even 
gained a little better grade than that. This finding was also positive, because expanding 
the number of tasks for each individual sup-part of the examination did not remarkably 
affect average grades. 


Conclusions and Implications 
This research pondered how the analysis of the assessment part of the sample 


technology course provided information on NDU’s technology education arrangements. 
Because this research utilized only on narrow part of the technology education and the 
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observations were made only in three successive years, the data is beyond of wider 
generalization. So even though the material described in the study is large (NV = 369) 
variation of parameters is limited. It seems that a summative test of 5 elements gave 
students more options to show their gained knowledge. But this is only a recommendation 
for further research where students’ profile would guide what type of efforts will be 
selected to show knowledge and skill in the assessment. The findings include the option 
to export this form based (or paper and pen type) examination into a portal- based variant 
for a same type of course. 
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